Algebraic Overcomplete Independent Component Analysis

Algebraic Independent Component Analysis (AICA) is a new ICA algorithm that exploits algebraic operations and vectordistance measures to estimate the unknown mixing matrix in a scaled algebraic domain. AICA possesses stability and convergence properties similar to earlier proposed geometic ICA (geo-ICA) algorithms, however, the choice of the proposed algebraic measures in AICA has several advantages. Firstly, it leads to considerable reduction in the computational complexity of the AICA algorithm as compared to similar algorithms relying on geometric measures making AICA more suitable for online implementations. Secondly, algebraic operations exhibit robustness against the inherent permutation and scaling issues in ICA, which simplifies the performance evaluation of the ICA algorithms using algebraic measures. Thirdly, the algebraic framework is directly extendable to any dimension of ICA problems exhibiting only a linear increase in the computational cost as a function of the mixing matrix dimension. The algorithm has been extensively tested for over-complete, under-complete and quadratic ICA using unimodal super-gaussian distributions. For other less peaked distributions, the algorithm can be applied with slight modifications. In this paper we focus on the overcomplete case, i.e., more sources than sensors. The overcomplete ICA is solved in two stages, AICA is used to estimate the rectangular mixing matrix, which is followed by optimal source inferencing using L1 norm based interior point LP technique. Further, some practical techniques are discussed to enhance the algebraic resolution of the AICA solution for cases where some of the columns of the mixing matrix are algebraically “close” to each other. Two illustrative simulation examples have also been presented.